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TUTORIAL 


SURVEYING  COST  GROWTH 

(apt  Vince  Sipple,  USAF,  Mai  Edward  "Tony"  White,  USAF, 
and  Maj  Michael  Greiner,  USAF 

Cost  growth  that  weapon  systems  incur  throughout  their  acquisition  life  cycle 
concerns  those  who  work  in  the  acquisition  environment.  One  way  to  reduce 
the  amount  of  unexpected  cost  growth  is  to  develop  better  cost  estimates.  In 
attaining  better  cost  estimates  though,  it  is  often  helpful  to  understand  and 
account  for  potential  cost  drivers.  Several  cost  studies,  some  of  which  specifically 
focus  on  the  aircraft  industry,  have  been  performed  documenting  and 
investigating  these  growth  factors.  Overviews  of  these  various  cost  growth 
studies  are  presented  as  other  tools  for  the  cost  estimators  and  program 
managers. 


The  cost  growth  that  major  weapon 
systems  incur  represents  a  major 
management  challenge.  A  1993 
study  by  RAND  cites  that  by  the  time  a 
Department  of  Defense  (DoD)  Acquisi¬ 
tion  Category  (ACAT)  I  program  com¬ 
pletes  the  production  and  fielding  phase 
of  acquisition,  it  will  experience  an 
average  cost  growth  of  approximately  20 
percent  from  its  initial  estimate  (Drezner, 
Jarvaise,  Hess,  Hough,  &  Norton,  1993). 

During  the  early  eighties,  the  Reagan 
administration  recognized  two  ways  to 
control  the  problem  of  cost  growth  — 
perform  cost/requirements  tradeoffs  when 
costs  grow  and  create  better  estimates 
(Office  of  the  Under  Secretary  of  Defense, 
1981).  Since  then,  high-level  DoD 
management  personnel  continue  to  seek 
better  ways  of  controlling  cost  growth. 

While  program  managers  shoulder  the 
burden  of  controlling  cost  growth,  the 


second  method  promoted  by  the  Reagan 
administration  for  addressing  cost  growth, 
creating  more  realistic  estimates,  pertains 
more  directly  to  the  cost  estimating  com¬ 
munity  (Office  of  the  Under  Secretary  of 
Defense,  1981).  The  impediments  to 
creating  more  realistic  estimates  prima¬ 
rily  stem  from  the  many  uncertainties  that 
estimators  encounter  during  their  data 
collection  efforts.  The  Office  of  the 
Secretary  of  Defense  (OSD)  Cost  Analysis 
Improvement  Group  (CAIG)  gives  guide¬ 
lines  for  documenting  cost  estimating 
uncertainty  for  DoD  system  acquisition 
programs. 

First,  they  mandate  that  “areas  of  cost 
estimating  uncertainty  will  be  identified 
and  quantified”  (Department  of  Defense 
[DoD],  1992,  p.  22).  Programs  must 
document  this  uncertainty  in  the  Cost 
Analysis  Requirements  Document 
(CARD).  Second,  the  CAIG  prescribes 
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“the  use  of  probability  distributions  or 
ranges  of  cost”  to  quantify  uncertainty 
(DoD,  1992,  p.  22).  Third,  they  ask  that 
the  uncertainty  estimated  be  “attributable 
to  estimating  errors”  (DoD,  1992,  p.  22). 
For  instance,  they  list  as  such  examples, 
performance  and  weight  characteristics, 
new  technology,  manufacturing  initia¬ 
tives,  inventory  objectives,  schedules,  and 
financial  condition  of  the  contractor. 

DoD  procedures  also  provide  for  con¬ 
tingency  estimation  and  sensitivity  analy¬ 
sis,  giving  the  estimator  the  option  to  either 


include  or  exclude  a  contingency  amount. 
If  the  estimator  includes  contingency 
amount,  he  must  give  the  reason  for  the 
contingency  estimate  as  well  its  rationale. 
In  addition,  he  must  “include  an 
assessment  of  the  likelihood  that  the  cir¬ 
cumstances  requiring  the  contingency  will 
occur”  (DoD,  1992,  p.  22). 

To  better  address  uncertainty  in  the 
estimating  process,  the  defense  department 
sponsors  much  research.  For  this  article’s 
preparation,  we  reviewed  past  growth  stud¬ 
ies  in  the  literature  and  highlight  those  here 


Table  1.  Consolidated  Results  of  Reviewed  Studies 


Study 

Main  Findings 

BMDO 

•Average  RDT&E  cost  growth  is  21%. 

•  Average  Production  cost  growth  is  1 9%. 

•  Only  7  to  16%  of  programs  complete  at  or  below  target  cost. 

•  Programs  with  lower  dollar  value  have  greater  likelihood  of  cost  growth. 

RAND 

•  On  average  smaller  programs  have  more  cost  growth. 

•  RDT&E  funds  tend  to  experience  more  cost  growth  than  production  funds. 

•  Programs  maturity  affects  cost  growth.  Longer  implies  greater  likelihood. 

•  New-start  programs  over  modification  programs  have  more  cost  growth. 

NAVAIR 

•  Cost  growth  varies  across  different  cohorts  (grouped  by  different  estimate 
categories). 

•  No  statistical  significance  in  overall  cost  growth  due  to  program  size. 

•Acquisition  changes  since  the  end  of  the  Cold  War  have  lead  to  less  cost  growth. 

•  Cost  growth  may  vary  by  commodity. 

CHRISTENSEN 

&TEMPLIN 

•  Management  reserve  budgets  sensitive  to  contact  category  (fixed-price  higher 
than  cost  reimbursable). 

•  MR  budgets  do  not  differ  between  production  and  RDT&E  contracts. 

ESKEW 

•  Weight,  speed,  production  rate,  and  time  explain  more  than  90  percent  of  the 
variation  in  cost  growth  of  fighter  aircraft. 

IDA 

•  Urgency  of  the  program,  difficulty  of  technology,  and  degree  of  testing  affect  cost 
growth. 

•A  relationship  between  cost  growth  and  schedule  growth  in  both  the  development 
and  the  production  phases  exist. 

RAND-JSF 

•  Differences  between  the  competitive  and  non-competitive  development  and 
procurement  cost  growth  factors  are  not  statistically  significant. 
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that  we  feel  provides  the  cost  estimating 
community  with  an  overall  synopsis  of 
available  research.  We  list  and  explain 
those  in  detail  in  the  remaining  sections. 
For  ease  of  review  we  have  summarized 
their  major  findings  in  Table  1. 


Studies  of  Cost  Growth 
in  DoD  Acquisitions 


Ballistic  Missile  Defense 
Organization  Study 

A  recent  Ballistic  Missile  Defense 
Organization  (BMDO)  cost  growth  study 
provides  insight  into  the  nature  of  cost 
growth  in  DoD  programs.  Using  an  inter¬ 
nal  BMDO  database  of  programs  (created 
from  a  subset  of  the  Selected  Acquisition 
Report  [SAR]  database),  BMDO  finds  that 
Research  and  Development,  Test,  and 
Evaluation  (RDT&E)  cost  growth  averages 
21  percent  while  that  of  production  aver¬ 
ages  19  percent  (Coleman,  Summerville, 
DuBois,  &  Myers,  2000).  The  study  also 
shows  that  from  7  to  16  percent  of  pro¬ 
grams  complete  at  or  below  the  target  cost. 
Additionally,  data  from  the  study  suggests 
that  the  lower  the  dollar  value  of  a  pro¬ 
gram,  the  greater  likelihood  of  a  large  cost 
growth  factor.  Despite  this  trend,  though, 
the  authors  do  not  provide  any  statistical 
tests  to  explore  this  possibility. 

The  BMDO  researchers  further  note  that 
as  a  program  progresses,  cost  estimators 
revise  their  estimates,  thereby  reducing  the 
amount  of  estimated  risk  and  increasing 
the  amount  of  realized  risk.  Under  the  as¬ 
sumption  of  unbiased  risk  estimates,  one 
expects  realized  risk  to  equal  estimated  risk 
on  average,  given  a  large  sample.  How¬ 
ever,  the  study  shows  that  the  risk  portion 
of  the  estimate  decreases  more  slowly  than 


the  rest  of  the  estimate  increases.  This  evi¬ 
dences  a  general  trend  of  underestimating 
risk  (Coleman  et  al.,  2000). 

RAND  Study  (1993) 

The  study  canvassing  an  extensive  array 
of  information  is  arguably  that  of  over  100 
data  points  performed  by  RAND.  Like  the 
BMDO  study,  RAND  uses  data  from  the 
SAR  reports,  and  RAND  focuses  on  the 
seven  categories  of  cost  variance  that  the 
SAR  contains:  quantity,  economic,  sched¬ 
ule,  engineering,  estimating,  support,  and 
other.  In  this  study,  RAND  finds  that  eco¬ 
nomic  and  quantity  changes  have  the  great¬ 
est  impact  on  cost  growth.  However,  RAND 
excludes  them  from  their  study  because 
these  two  factors  are  part  of  the  assump¬ 
tions  of  a  cost  estimate  initially. 

The  RAND  study  goes  on  to  relay  sev¬ 
eral  other  factors  that  relate  to  cost  growth. 
Like  the  BMDO  study,  RAND  finds  an  ap¬ 
parent  difference  in  cost  growth  based  on 
program  size.  That  is, 
smaller  programs  have 
on  average  more  cost 
growth  than  larger 
ones.  The  RAND  re¬ 
searchers  postulate  as 
the  reason  behind  this 
difference  the  greater 
level  of  management 
scrutiny  that  higher  dol¬ 
lar  programs  receive. 

This  stands  to  reason  considering  more 
management  scrutiny  should  translate  into 
better  cost  management. 

The  authors  of  the  RAND  study  offer 
another  possible  explanation  for  the  dif¬ 
ference  in  cost  growth  of  the  smaller  pro¬ 
grams,  “R&D  costs  are  a  large  portion  of 
total  costs  and  tend  to  incur  more  cost 
growth”  (Drezner  et  al.,  1993,  p.  49). 


"data  from  the 
study  suggests 
that  the  lower  the 
dollar  value  of  a 
program,  the 
greater  likelihood 
of  a  large  cost 
growth  factor." 
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In  other  words,  smaller  programs  are 
disproportionately  smaller  in  procure¬ 
ment  than  in  RDT&E.  Since  RDT&E 
funds  tend  to  experience  more  cost 
growth  than  procurement  funds,  the 
same  percent  RDT&E  cost  growth  in  a 
smaller  program  pushes  the  overall  pro¬ 
gram  percentage  cost  growth  higher 
than  a  large  program  counterpart. 

Program  maturity  also  factors  largely 
in  program  cost  growth.  The  RAND  study 
notes  that  “on  average,  cost  growth  in¬ 
creases  by  2.2  percent 
per  year  above  infla¬ 
tion  because  of  the  ef¬ 
fects  of  maturity” 
(Drezner  et  al.,  1993, 
p.  49).  RAND  empha¬ 
sizes  the  importance  of 
these  two  factors 
above  other  factors  in 
the  statement,  “pro¬ 
gram  size  and  maturity 
can  dominate  other 
factors  affecting  cost 
growth  outcomes  and  so 
must  be  considered  in 
both  the  analysis  and  the 
interpretation  of  re¬ 
sults”  (Drezner  et  al.,  1993,  p.  49). 

The  RAND  study  elucidates  the  impact 
of  new-start  programs  versus  modification 
programs,  finding  that  on  average,  the 
new-start  programs  experience  more  cost 
growth  than  modification  programs.  The 
RAND  study  also  finds  longer  programs 
to  have  more  cost  growth  than  shorter 
ones.  This  relationship  proves  intuitive: 
each  year  brings  the  opportunity  for  more 
cost  growth.  Interestingly,  they  find  no 
relationship  between  planned  length  and 
cost  growth  nor  between  schedule  slip 
and  cost  growth  (Drezner  et  al.,  1993). 


"While  the 
prototyping 
probably  does 
significantly 
reduce  risk,  it 
apparently  does 
not  reduce  it  to 
the  extent  that 
would  make  a 
prototyped 
program  have 
less  cost  growth 
than  a  non- 
prototyped 
program." 


Finally,  RAND  finds  a  correlation 
between  prototyping  and  cost  growth: 

We  compared  the  cost  outcomes 
of  prototyping  and  nonproto¬ 
typing  programs,  expecting  to 
find  that  a  prototype  development 
strategy  contributes  to  cost  con¬ 
trol  through  reduction  of  uncer¬ 
tainty.  Interestingly,  programs 
that  included  prototyping  had  a 
relatively  higher  cost  growth.  This 
result  may  be  due  in  part  to  the 
timing  of  the  prototype  phase 
within  the  context  of  the  overall 
program  schedule,  since  earlier 
prototyping  makes  data  available 
earlier,  thus  potentially  affecting 
the  baseline  cost  estimate  at  the 
time  of  EMD  start.  Our  results  are 
consistent  with  this  notion.  It  may 
also  be  true  that  prototyping  was 
conducted  for  programs  with  rela¬ 
tively  higher  degrees  of  technical 
uncertainty,  a  hypothesis  that 
deserves  further  exploration. 
(Drezner  et  al.,  1993,  p.  51) 

Given  that  DoD  prescribes  risk  reduc¬ 
tion  such  as  prototyping  for  riskier  pro¬ 
grams,  RAND’s  assessment  rings  true. 
While  the  prototyping  probably  does 
significantly  reduce  risk,  it  apparently 
does  not  reduce  it  to  the  extent  that  would 
make  a  prototyped  program  have  less  cost 
growth  than  a  non-prototyped  program. 

NAVAIR  Study 

Naval  Air  Systems  Command 
(NAVAIR)  presents  its  most  recent  study 
on  cost  growth  at  the  2001  DoD  Cost 
Analysis  Symposium,  corroborating 
some  of  the  results  of  previous  studies. 
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Here  again,  SAR  data  are  used.  As  part 
of  their  analysis,  they  explore  the  pos¬ 
sible  need  for  “cohort  tracking”  when 
analyzing  cost  growth  (Dameron,  Pullen, 
Summerville,  Coleman,  &  Snead,  2001). 

By  cohort  tracking,  the  NAVAIR 
team  refers  to  the  grouping  of  cost 
growth  according  to  certain  program¬ 
matic  characteristics  that  relate  to  com¬ 
mon  patterns  of  cost  growth.  The  team 
divides  program  cost  growth  into  five 
categories  or  cohorts  —  RDT&E  cost 
growth  for  programs  with  a  planning 
estimate  (PE)  and  a  development  esti¬ 
mate  (DE);  RDT&E  cost  growth  for  pro¬ 
grams  with  a  DE  only;  procurement  cost 
growth  for  programs  with  a  PE,  a  DE, 
and  a  production  estimate  (PdE);  pro¬ 
curement  cost  growth  for  programs  with 
a  DE  and  a  PdE  only;  and  procurement 
cost  growth  for  programs  with  a  DE 
only  (Dameron  et  al.,  2001). 

The  three  different  estimates  (PE,  DE, 
and  PdE)  are  the  baseline  estimates  that 
correspond  to  each  of  the  major  decision 
points  in  Milestone  A,  B,  and  C  respec¬ 
tively.  NAVAIR  uses  the  five  cohorts  con¬ 
sisting  of  the  different  types  of  estimates 
to  categorize  the  cost  growth,  because  the 
use  of  those  mixes  of  cost  estimates  relate 
to  different  types  of  program  structures, 
which  might  represent  distinct  populations 
with  distinct  cost  growth  patterns. 

After  looking  at  318  DoD  programs, 
the  cohort  study  results  show  that  the 
PE  and  DE  cohort  has  an  average  of  30 
percent  RDT&E  cost  growth;  the  DE- 
only  cohort  has  an  average  of  25  per¬ 
cent  RDT&E  cost  growth;  the  PE,  DE, 
and  PdE  cohort  has  an  average  of  35 
percent  procurement  cost  growth;  the 
DE  and  PdE  cohort  has  an  average  of 
25  percent  procurement  cost  growth;  and 


the  DE-only  cohort  has  an  average  of  15 
percent  procurement  cost  growth.  The 
sample  sizes  are  25,  140,  6,  53,  and  94 
respectively  (Dameron  et  al.,  2001).  The 
NAVAIR  group  indicates  that  the  “results 
are  tentative,”  but  their  findings  suggest 
differences  in  cost  growth  from  one  co¬ 
hort  to  another. 

In  particular,  they  point  out  that,  in  their 
study,  “programs  with  a  Program  Defini¬ 
tion  and  Risk  Reduction  (PDRR)  phase 
have  more  growth”  (Dameron  et  al., 
2001,  p.  11).  The  pur¬ 
pose  of  PDRR  is  to  re¬ 
duce  risk,  so  programs 
with  PDRR  naturally 
have  a  lot  of  uncer¬ 
tainty,  and  quantifying 
the  costs  of  such  a  pro¬ 
gram  should  be  more 
difficult  than  for  less 
risky  programs.  In  ad¬ 
dition,  the  natural  cor¬ 
relation  between  pro¬ 
grams  with  a  PDRR 
phase  and  programs 
with  a  prototyping  ef¬ 
fort,  leads  one  to  expect  similar  results  as 
the  two  relate  to  cost  growth.  Finally,  pro¬ 
grams  with  a  PDRR  phase  tend  to  have  a 
longer  RDT&E  funding  year  stream  than 
those  without  PDRR.  This  meshes  with 
RAND’s  finding  that  longer  programs 
tend  to  have  higher  cost  growth.  Thus, 
consistency  exists  in  the  findings  show¬ 
ing  that  programs  with  risk  reduction  ef¬ 
forts  tend  toward  higher  cost  growth. 

The  NAVAIR  study  also  analyzes  cost 
growth  correlations  between  program 
phases  and  between  the  RDT&E  and  pro¬ 
curement  appropriations.  The  study 
finds  a  significant  correlation  between 
RDT&E  cost  growth  in  the  PDRR  phase 


"By  cohort 
tracking,  the 
NAVAIR  team 
refers  to  the 
grouping  of  cost 
growth  according 
to  certain 
programmatic 
characteristics 
that  relate  to 
common  patterns 
of  cost  growth." 
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and  RDT&E  cost  growth  in  the  EMD 
phase  and  also  finds  “significant  corre¬ 
lation  between  procurement  growth 
during  the  EMD  and  production  phases” 
(Dameron  et  al.,  2001,  p.  14).  Finally, 
the  study  uncovers  a  significant  corre¬ 
lation  between  appropriations  such  that, 
during  EMD,  when  the  RDT&E  appro¬ 
priation  experiences  cost  growth,  so 
does  the  procurement  appropriation 
(Dameron  et  al.,  2001). 

As  a  third  area  of  study,  the  NAVAIR 
group  analyzes  how  program  size  affects 
cost  growth.  Unlike  the  BMDO  and 
RAND  studies,  the 
NAVAIR  team  performs 
statistical  comparisons 
that  reveal  that  high  and 
low  dollar  programs 
have  identical  distribu¬ 
tions  despite  “a  trend  of 
more  high  end  extrema 
in  the  smaller  size 
classes”  (Dameron  et  al., 
2001,  p.  21).  To  explain  the  difference  in 
the  extrema,  they  reason  that,  “high  risk 
programs  may  be  terminated  earlier  if 
large,  but  tolerated  if  small”  (Dameron  et 
al.,  2001,  p.  21).  Thus,  inferential  statis¬ 
tics  suggests  no  difference  in  the  overall 
cost  growth  based  on  size. 

Next,  NAVAIR  studies  the  effects  of  the 
era  in  which  an  acquisition  terminates. 
The  team  uses  “DoD  programs  with  DE 
only  from  the  RAND  93  dataset,  NAVAIR 
programs  with  DE  only  from  the  SAR  00 
dataset,  and  NAVAIR  programs  with  DE 
only  from  the  Contract  dataset  (RDT&E 
only)”  (Dameron  et  al.,  2001,  p.  23).  Thus, 
they  use  three  separate  data  sets,  two  of 
their  own  compilation  and  the  RAND  93 
dataset.  The  group  studies  the  effects  of 
two  eras  —  pre-1986  and  post-1986. 


"NAVAIR's 
results  suggest 
that  acquisition 
changes  since  the 
end  of  the  Cold 
War  lead  to  less 
cost  growth." 


They  choose  1986  as  a  dividing  point, 
because  that  year  marks  the  last  year  of 
the  Reagan  arms  buildup  (Dameron  et  al., 
2001).  The  team  performs  t-tests  to  deter¬ 
mine  if  the  two  eras  differ  statistically. 
They  find  the  following  results: 

•  RAND  93:  The  means  of  programs 
through  1986  and  those  after  1986  did 
show  a  statistical  difference  for 
RDT&E,  but  not  for  procurement. 

•  SAR  00:  The  means  of  programs 
through  1986  and  those  after  1986 
did  show  a  statistical  difference  for 
procurement,  but  not  for  RDT&E. 

•  Contract:  The  means  of  programs 
through  1986  and  those  after  1986  did 
not  show  a  statistical  difference  for 
RDT&E.  (Dameron  et  al.,  2001,  p.  31) 

The  team  concludes  that  their  “analy¬ 
sis  supports  a  decline  in  cost  growth  fac¬ 
tor  (CGF)  over  time”  (Dameron  et  al., 
2001,  p.  32).  They  mention  that  these 
results  differ  from  previous  studies  per¬ 
haps  because  of  differences  in  the  num¬ 
ber  of  data  points  or  dates  of  era  division 
(Dameron  et  al.,  2001). 

NAVAIR’s  results  suggest  that  acqui¬ 
sition  changes  since  the  end  of  the  Cold 
War  lead  to  less  cost  growth.  Although 
difficult  to  pinpoint  the  reason  for  this 
decline  in  cost  growth,  logic  leads  to 
candidate  scenarios.  The  draw-down  in 
military  spending  after  the  Cold  War  pro¬ 
duces  an  environment  in  the  government 
where  meeting  cost  goals  becomes  more 
important  for  program  survival.  So  per¬ 
haps  the  ensuing  emphasis  on  better  es¬ 
timating  improves  base-line  estimates 
from  which  cost  growth  is  measured. 
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Improvements  in  the  estimating  profes¬ 
sion  might  also  play  a  part  in  improv¬ 
ing  base-line  estimates. 

NAVAIR  also  studies  differences  be¬ 
tween  commodities  and  their  relation  to 
cost  growth.  The  team  looks  at  all  three 
databases,  but  limits  the  data  to  20  RAND 
93  programs,  1 1  SAR  00  programs,  and 
2 1  contract  data  programs.  They  conclude 
that  missile  programs  experience  higher 
cost  growth  during  RDT&E  than  either 
electronic  or  aircraft  programs.  This  result 
suggests  that  cost  growth  may  vary  by 
commodity. 

The  NAVAIR  team  further  compares 
RDT&E  cost  growth  in  small  programs 
(less  than  one  billion  dollars  in  RDT&E) 
as  portrayed  through  the  SAR  2000  data 
versus  the  NAVAIR  contract  database. 
Their  analysis  shows  that  the  results  from 
the  two  databases  do  not  significantly  dif¬ 
fer  (Dameron  et  al.,  2001).  They  conclude 
that  potential  exists  to  use  either  database 
to  study  cost  growth.  Intuitively,  cost 
growth  from  a  contract  only  perspective 
miiTors  that  of  an  overall  program  cost 
perspective,  because  the  vast  majority  of 
program  dollars  apply  to  contracts. 

Christensen  and  Tempun  Study 

David  Christensen  and  Carl  Templin  re¬ 
search  cost  growth  using  the  Defense  Ac¬ 
quisition  Executive  Summary  (DAES) 
database  and  arrive  at  potentially  useful 
findings  in  the  search  for  predictors  of  cost 
growth.  The  DAES  database  contains  con¬ 
tractor  information  organized  according 
to  the  rules  of  Earned  Value  Management, 
a  process  by  which  the  government 
monitors  the  cost  and  schedule  perfor¬ 
mance  of  contracts  against  baseline  fig¬ 
ures  (Christensen  &  Templin,  2000). 


The  researchers  consider  “hundreds 
of  DoD  defense  acquisition  contracts 
from  1975  through  1998”  in  a  hypoth¬ 
esis  testing  scenario  focused  on  the  na¬ 
ture  of  management  reserve  (MR)  bud¬ 
gets  (Christensen  &  Templin,  2000). 
DoD  characterizes  an  MR  budget  as  “a 
reserve  for  uncertainties  related  to  in¬ 
scope  but  unforeseen  work”  (DoD, 
1997,  p.  12).  MR  budgets,  because  they 
represent  the  contractors’  assessment  of 
risk  for  acquisition  programs,  can  pro¬ 
vide  useful  insight  into  the  overall  risk 
assessment  that  DoD  uses  in  its  bud¬ 
geting  process. 

Christensen  and  Templin  recognize 
that  many  factors  affect  the  develop¬ 
ment  of  a  contractor’s  MR  budget,  and 
that  the  “achievability  of  a  budget  de¬ 
pends  on  how  the  bud¬ 
gets  are  established” 

(Christensen  &  Templin, 

2000,  p.  195).  Thus, 
overruns  can  vary  de¬ 
pending  on  factors 
such  as  differing  meth¬ 
ods,  abilities,  and  mo¬ 
tivations  of  those  who 
set  the  MR  budgets 
(Christensen  &  Templin, 

2000).  A  1998  survey  of 
300  DoD  risk  analysis 
professionals  (U.S.  Aerospace  Cost  Risk 
Analysis  Survey,  2000)  supports  this 
statement  by  displaying  the  following 
variety  of  perspectives  on  risk  analysis 
extant  within  government  and  contrac¬ 
tor  circles. 

•  27  percent  of  analyses  perform  the 
risk  assessment  separately  from  the 
cost  estimate. 


"They  [the 
team]  conclude 
that  missile 
programs 
experience 
higher  cost 
growth  during 
RDT&E  than 
either  electronic 
or  aircraft 
programs." 
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•  26  percent  of  program  managers  do 
not  accept  risk  assessment  at  all,  not 
even  slightly. 

•  32  percent  of  the  risk  assessments  do 
not  involve  finance  or  estimating. 

•  38  percent  of  cost  risk  analysts  have 
received  no  training,  either  formal  or 
informal. 

•  44  percent  of  risk  ranges  are  intuitive 
judgments,  without  historical  data  or 
guided-survey. 

•  69  percent  of  variable  distributions  are 
triangular. 

•  18  percent  of  unfavorable  assessments 
are  ignored,  as  managers  stay  the 
course. 

In  addition,  Christensen  and  Templin 
(2000)  note  that  contractors  should  pro¬ 
vide  greater  MR  budgets  for  riskier 
projects.  The  authors  characterize  the 
development  phase  of  acquisition  as  more 
uncertain  than  the  production  phase,  and 
they  characterize  price  contracts  as  more 
uncertain  than  cost-reimbursement  con¬ 
tracts  (Christensen  &  Templin,  2000). 
From  this  awareness  of  the  diversity  of  risk 
analysis,  Christensen  and  Templin  per¬ 
form  hypotheses  testing  to  realize  the 
following  results: 

The  amount  of  an  MR  budget  is 
sensitive  to  contract  category 
(cost-reimbursable  versus  fixed- 
price),  and  the  managing  service. 
With  regard  to  contract  category, 
the  median  MR  percent  on  fixed- 


price  contracts  is  significantly 
greater  than  the  median  MR  per¬ 
cent  on  cost  reimbursable  con¬ 
tracts.  This  is  consistent  with  the 
expectation  that  contracts  with 
more  risk  to  the  contractor  have  a 
larger  MR  budget.  We  do  not 
know  why  MR  budgets  differ 
across  the  three  services.  Possible 
explanatory  factors  include  differ¬ 
ences  in  the  weapon  systems  pur¬ 
chased  by  each  service,  and  the 
contractors  that  build  the  systems. 
(Christensen  &  Templin,  2000, 
p.  204) 


With  regard  to  the  acquisition  phase, 
the  researchers  do  not  find  that  the  MR 
budget  differs  between  production  and 
RDT&E  contracts  (Christensen  &  Temp¬ 
lin,  2000).  Christensen  and  Templin 
(2000)  shed  light  into  the  way  that  con¬ 
tractors  manage  risk  through  MR  budgets. 
The  relationship  between  contract  type 
and  the  MR  budget  stands  to  reason,  since 
certain  contract  types  place  more  risk  on 
the  contractor  than  others.  The  MR  bud¬ 
get  insensitivity  to  acquisition  phase 
differs  from  the  government’s  perspective 
that  RDT&E  efforts  are  more  risky  than 
procurement  efforts;  this  difference  reem¬ 
phasizes  the  importance  of  using  contract 
type  to  instigate  contractor  behavior  that 
best  advances  government  objectives.  The 
sensitivity  to  managing  service  proves 
enigmatic.  While  possible  that  significant 
differences  exist  in  the  way  each  service 
manages  its  contracts,  it  may  be  that  some 
other  variable  or  variables,  highly  corre¬ 
lated  with  managing  service  triggers  this 
sensitivity. 
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Cost  Growth  Specific  to 
the  Aerospace  Industry 


Eskew  Study 

To  find  the  true  rate  of  cost  growth  of 
fighter  aircraft  over  time,  Henry  Eskew 
runs  a  multiple  linear  regression  of  17 
tactical  aircraft  from  1950  through  1980 
(Eskew,  2000).  He  normalizes  his  data  for 
production  quantity  by  using  the  estimated 
100th  production  unit  cost,  and  he  nor¬ 
malizes  his  data  for  inflation  by  applying 
the  appropriate  DoD  inflation  indices  to 
convert  his  data  to  constant  year  (CY) 
1990.  El sing  the  logarithm  of  cost  as  his 
response  variable,  he  finds  weight,  speed, 
production  rate,  and  time  as  statistically 
significant  predictor  variables  that  explain 
“more  than  90  percent  of  the  variation  in 
cost”  (Eskew,  2000,  pp.  211-212).  He  also 
determines  that,  as  a  sole  predictor,  time 
explains  about  40  percent  of  the  cost 
variation  (Eskew,  2000). 

Although  useful  as  a  literature  review, 
one  must  note  some  limitation  of  the 
Eskew  study’s  applicability.  First,  the 
study  looks  at  a  limited  amount  of  data 
from  a  limited  perspective.  It  only  con¬ 
siders  tactical  aircraft  in  its  search  for 
predictors,  and  it  only  has  17  data  points. 
In  addition,  this  research  lacks  currency, 
spanning  the  period  from  1950  through 
1980,  and  seeks  to  explain  cost  growth  as 
overall  increases  in  unit  cost  measured 
from  previous  programs  over  time  (Eskew, 
2000).  Most  of  the  research  we  consider 
heretofore  considers  cost  growth  within  a 
single  program  over  a  much  shorter  time 
period. 

Dr.  Eskew  (2000)  seeks  to  dispel  the 
myth  that  “no  systematic  relationship 
exists  between  the  characteristics  of  an 
aircraft  program  and  the  length  of  its 


development  cycle”  (Eskew,  2000,  p. 
210).  He  uses  the  same  normalization 
techniques  mentioned  earlier  for  inflation 
and  quantity;  however, 
he  includes  different 
aircraft,  adding  non- 
tactical  fixed  wing  air¬ 
craft,  and  removing 
non-fixed  wing  air¬ 
craft  (Eskew,  2000). 

The  results  of  his  17 
data-point  regression 
show  that  unit  flya¬ 
way  cost  predicts  ap¬ 
proximately  60  per¬ 
cent  of  the  variance  in 
the  length  of  the  de¬ 
velopment  program: 
this  predictive  ability 
increases  to  70  percent  when  a  dummy 
variable  is  added  indicating  whether  or 
not  a  program  has  inherited  a  significant 
amount  of  technology  from  a  previous 
program  (Eskew,  2000).  Overall,  the 
Eskew  study  highlights  a  correlation  be¬ 
tween  aircraft  physical  and  functional 
characteristics  and  production  costs,  and 
between  program  schedule  and  produc¬ 
tion  costs. 

IDA  Study 

The  Institute  for  Defense  Analyses  (IDA) 
performs  an  analysis  on  cost  and  schedule 
growth  of  tactical  missiles  and  tactical  air¬ 
craft  in  1994  with  the  goal  of  finding  pat¬ 
terns  of  cost  growth  and  the  reasons  for 
the  cost  growth  (Tyson,  Harmon,  &  Utech, 
1994).  Within  the  group  of  20  tactical  mis¬ 
siles  investigated,  the  IDA  group  finds  that 
only  two  stay  within  their  schedule,  with 
one  program  slipping  by  as  much  as  180 
percent.  They  also  find  that  only  two  pro¬ 
grams  stay  within  budget,  while  the  two 


"Overall,  the 
Eskew  study 
highlights  a 
correlation 
between  aircraft 
physical  and 
functional 
characteristics 
and  production 
costs,  and 
between  program 
schedule  and 
production  costs." 
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worst  performers  exceed  their  budgets 
by  a  factor  of  two  (Tyson  et  al.,  1994). 
The  researchers  of  IDA  examine  the 
characteristics  of  the  programs  with  the 
highest  and  lowest  schedule  and  cost 
growth.  From  their  study,  they  find  that: 

[Missile]  programs  that  employed 
a  high  degree  of  concurrency,  that 
had  to  be  dual-sourced  for  tech¬ 
nical  reasons  or  that  were  dual- 
sourced  at  less  than  full  rate,  had 
high  cost  growth.  In  one  case,  the 
threat  of  competition  appeared  to 
reduce  costs.  (Tyson  et  ah,  1994, 
p.  S-2) 

The  results  from  aircraft  programs  do 
not  vary  as  much.  The  authors  suggest 
closer  management  scrutiny  and  “protec¬ 
tion  from  schedule  stretch”  as  reasons  for 
the  more  consistent  cost  growth  in  aircraft 
programs  (Tyson  et  al.,  1994,  p.  S-2).  Two 
aircraft  programs  suffer  elongated  produc¬ 
tion  schedules,  but  do  not  experience  high 
production  cost  growth.  The  authors  theo¬ 
rize  that  generally  extending  production 
incites  cost  growth;  however,  in  these 
cases  the  existence  of  other  DoD  contracts 
cushions  the  impact  of  the  adjusted  sched¬ 
ules.  The  authors  identify  the  F/A-18  as 
the  program  with  the  highest  cost  growth. 
They  theorize  that  late  engineering 
changes  incite  the  high  cost  growth  (Tyson 
et  al.,  1994). 

The  study  considers  whether  modifica¬ 
tion  programs  have  lower  cost  growth  than 
new  start  programs.  The  one  aircraft  in 
their  sample  that  is  a  modification  does 
in  fact  experience  low  cost  growth.  The 
team  finds  that  missile  modification  pro¬ 
grams  vary  greatly  in  the  cost  growth  they 


experience.  They  cite  the  fact  that  most 
missile  modifications  affect  the  expensive 
guidance  and  control  system  of  the  missile 
as  a  possible  reason  for  this  inconsistency 
(Tyson  et  al.,  1994). 

The  researchers  further  find  that  the 
urgency  of  the  program,  the  difficulty  of 
the  technology,  the  amount  of  concur¬ 
rency,  and  the  degree  of  testing  all  seem 
to  affect  cost  growth  (Tyson  et  al.,  1994). 
From  these  results,  the  IDA  researchers 
discover  a  relationship  between  cost 
growth  and  schedule  growth  in  both  the 
development  and  the  production  phases 
(Tyson  et  al.,  1994).  They  find  that  quan¬ 
tity  increases  during  development  largely 
drive  development  schedule  growth.  The 
authors  mention  “the  need  to  produce 
more  items  for  testing  than  planned”  as 
the  reason  for  the  increase  in  quantity 
(Tyson  et  al.,  1994,  p.  S-6). 

It  is  not  clear  whether  failed  tests  drive 
the  need  to  produce  more  test  units  or  sim¬ 
ply  uncertainty  in  the  planning  process 
drove  the  need  to  produce  more  test  units. 
Either  way,  one  can  draw  a  link  back  to 
technical  risk  from  the  need  for  more  test 
units.  The  study  also  finds  that  whether  a 
missile  is  an  intercept  missile  and  the 
length  of  the  original  schedule  prove  use¬ 
ful  predictors  of  development  schedule 
growth. 

Lastly,  the  IDA  study  adds  depth  to  the 
study  of  cost  growth  by  using  multiple 
regression  to  arrive  at  predictive  formu¬ 
las.  Using  multiple  regression  brings  with 
it  the  benefit  of  taking  into  account  the 
multi-dimensional  interactions  that  inde¬ 
pendent  variables  have  with  dependent 
variables  that  can  prove  misleading  in 
simple  linear  regression  analysis  and 
hypothesis  testing. 
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RAND  Study  (2001) 

In  support  of  the  Joint  Strike  Fighter 
(JSF)  program,  RAND  studies  the  effect 
of  competition  on  the  amount  of  cost 
growth  that  occurs  in  both  the  RDT&E 
and  procurement  budgets  (Birkler,  Graser, 
Arena,  Cook,  Lee,  &  Lorell,  2001).  The 
researchers  analyze  14  programs  that  use 
competitive  strategies  and  44  programs 
that  do  not  (Birkler  et  al.,  2001).  They 
find  that  “the  results  are  mixed  and  the 
differences  between  the  competitive  and 
noncompetitive  development  [and  pro¬ 
curement]  cost  growth  factors... are  not 
statistically  significant  at  the  10-percent 
level”  (Birkler  et  al.,  2001,  p.  80). 

These  results  are  potentially  mislead¬ 
ing,  however,  in  that  they  represent  a 
program- wide  look,  rather  than  a  study  of 
individual  contracts.  In  other  words,  a  pro¬ 
gram  might  be  competitively  awarded 
initially,  but  at  a  certain  point  in  the  string 
of  contracts  that  make  up  a  program  one 
can  make  the  case  that  a  competitive  en¬ 
vironment  no  longer  exists.  Perhaps  a 
study  that  compared  individual  contracts 
(rather  than  entire  programs)  might  exact 
different  results. 

Conclusion 


In  this  article,  we  document  many  stud¬ 
ies  that  query  different  databases  using 
various  statistical  methods  in  the  effort  to 
explain  cost  growth  in  DoD  acquisition. 
We  consider  studies  of  overall  DoD 
acquisition  as  well  as  studies  that  focus 
on  a  particular  industry  within  the  DoD 
acquisition  landscape.  Further  research  is 
currently  being  conducted  to  examine/ 
investigate  how  best  to  model  cost  growth 


and  which  predictor  variables  should  be 
included  in  such  a  model.  From  these, 
appropriate  statistical  tools  and  method¬ 
ologies  could  then  be  made  available  to 
the  cost  community. 

This  article  applies  not  just  to  the  cost 
estimator,  but  also  to  the  entire  integrated 
process  team.  Knowledge  of  these  stud¬ 
ies  should  bring  to  bear  a  better  under¬ 
standing,  a  sharper  focus,  and  a  more 
efficient  approach  to  those  who  seek  to 
study  cost  growth  in  the  future.  It  should 
also  be  clear  to  the  estimator  that  more 
often  than  not,  estimates  will  be  low.  From 
the  many  studies  in  this  article,  the  esti¬ 
mator  should  find  some  insight  that  will 
lead  him  to  a  better  analysis  of  risk  to  com¬ 
pensate  in  part  for  that  tendency  to  under¬ 
estimate.  For  the  rest  of  the  integrated  pro¬ 
cess  team,  the  reality  is  that  cost  overruns 
can  kill  programs. 

As  mentioned  in  the  beginning  of  this 
article,  the  two  sides  of  the  solution  coin 
are:  more  realistic  baseline  estimate  (with 
accompanying  risk  dollars)  and  better  cost 
control.  Many  hindrances  might  stand  in 
the  way  of  achieving  more  realistic  esti¬ 
mates  and  better  cost  control.  Those  hin¬ 
drances  might  be  in  the  form  of  priority 
setting,  manpower  shortages,  lack  of  train¬ 
ing,  budget  cuts,  requirement  creep,  or  any 
number  of  other  issues  both  foreseen  or 
unforeseen.  This  article  focused  on  pro¬ 
viding  decision  makers  in  the  acquisition 
world  with  some  historical  insight  into 
what  research  has  been  done  regarding 
cost  growth.  With  knowing  what  has  been 
in  the  past,  it  provides  a  roadmap  to 
prevent  rework  as  well  as  fine  tune  future 
research  and  promote  ready-to-use 
analytical  tools. 
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